1. Field of Invention
This invention relates to methods of estimating the subjective quality of a multimedia communications system in which audio, voice or video is digitized, compressed, formed into packets, transmitted over a packet network and then re-assembled and decoded by a receiving system. Typical packet networks cause some packets to be lost or delayed which results in the quality of the decoded audio, voice or video being degraded and it is accordingly desirable to have some means of measuring or estimating the subjective or perceptual quality of the decoded audio, voice or video.
Emerging packet based voice networks, using technology such as Voice over IP (Internet Protocol), provide a more flexible and lower cost alternative to traditional telecommunications networks. They do however introduce some problems, notably increased variation in user perceived speech quality due to network impairments. The present invention relates to methods for estimating this variation in user perceived quality.
A Voice over IP system comprises two or more conversion points and a connecting network. A conversion point is a device that converts analog voice into packet format suitable for transmission over a network. A conversion point may be a device within a telephone switching system, a packet voice telephone, a personal computer running an applications program or other types of device. The following brief description may be referenced to the illustrative diagram shown in FIG. 1. At each conversion point the analog voice signal from the user""s telephone is converted to a digital form, divided into short segments, compressed, placed into an IP packet and then transmitted over the connecting network to the remote conversion point. Received voice packets are uncompressed, converted back to analog form and played to the user as an audible signal.
The connecting network relays the IP packets from one conversion point to another. The network is a shared resource and is carrying many other streams of packet data. This means that any given packet may be subject to impairments, for example:
(i) Delay, in which the time for the packet to get from one conversion point to the other conversion point causes delays in the apparent response from one user to the other
(ii) Packet loss, in which some of the packets are lost or arrive so late that they are discarded
(iii) Jitter, in which the arrival time of the packets varies
(iv) Distortion, due largely to the voice compression algorithm in use.
These impairments collectively cause the user perceived voice quality to vary considerably and hence Voice over IP service providers need a method for estimating the quality of service provided by their network (Voice Quality of Service).
2. Description of Prior Art
Prior art systems for measuring voice quality, as described by Douskalis (Hewlett Packard 2000), Royer (U.S. Pat. No. 5,710,791) and Di Pietro (U.S. Pat. No. 5,867,813), use centralized test equipment which samples the voice quality from various conversion points. A loop back condition is established at an conversion point, the test equipment transmits a known signal and then compares the received (looped back) signal with the original, thereby estimating delay, distortion and other impairments. This approach provides an accurate measure of voice distortion but only provides this measure for a sample conversion point and under the network conditions that existed at the time of the test. This approach is undesirable for continuous network monitoring as the frequent transmission of test messages increases the traffic in the network and reduces network performance.
Another approach currently used for estimating voice quality is to estimate the subjective performance of the voice connection using objectively measured parameters. Models such as the E-Model described by Johannesson, (IEEE Communications Magazine 1997), are able to produce R ratings which can be correlated to user perceived voice quality. This process is applied by a central management system which gathers statistics on noise and delay and then produces an estimate of voice quality. This method as described by Johannesson does not consider impairments typical of packets systems.
Experimental measurements of the effects of network impairments on packet voice quality are reported by Cermak (T1A1 contributions May 1999 and June 1998). Cermak considered the effects of average packet loss but did not consider the effects of the time varying nature of impairments on subjective quality.
The Quali.Net system marketed by ECTel comprises a central test system with additional remote test units. The remote test units are complex units that contain dedicated electronic circuitry and software and are constructed as separate items of test equipment that are externally attached to a Voice over IP system. The remote test units estimate voice quality on selected voice connections and report this to the central test system for diagnostic purposes. The high cost of these remote test units means that it is prohibitively expensive to install one for every voice connection and therefore only a small number are typically employed within a network. The Quali.Net system does not contain a statistical modeling process that analyses the burst nature of packet loss and its effects on subjective voice quality. The Quali.Net system does not compute the estimated subjective voice quality within the Voice over IP end system, cannot effectively monitor the voice quality at every port simultaneously and cannot provide per-call voice quality information that can be recorded within a call record database
The NeTrueQoS system marketed by NeTrue comprises a central test system with remote software agents. The software agents gather network statistics and report packet loss, jitter and delay back to the central system which computes an estimated voice quality. Said software agents are located within a Voice over IP Node, which comprises a piece of equipment that supports multiple Voice over IP ports. The NeTrueQoS system does not contain a statistical modeling process that analyses the burst nature of packet loss and its effects on subjective voice quality. The NeTrueQoS system does not compute the estimated subjective voice quality within the Voice over IP end system and therefore cannot effectively monitor the voice quality at every port simultaneously and cannot provide per-call voice quality information that can be recorded within a call record database.
Prior art systems for estimating voice quality based on measurements of network performance therefore suffer from a number of drawbacks:
(i) The use of the statistics gathered independently over a period of time does not reflect the time correlation between the statistics. If a high level of jitter coincides in time with a high level of packet loss then this will have a different subjective effect to the same impairments occurring at different times. Prior art centralized systems for estimating voice quality based on network statistics do not precisely correlate the times at which impairments occur and therefore do not accurately estimate voice quality.
(ii) Typical voice coding algorithms employed in packet voice systems compensate for lost packets by repeating the last packet, estimating the content of the lost packet or inserting noise. For single lost packets this approach is very effective and voice quality affected only slightly. When more than one subsequent packet is lost the voice coding algorithm will replay the last received packet multiple times, which is much more noticeable to the user. Prior art systems do not represent the way that bursts of lost packets affect voice quality and therefore do not accurately estimate voice quality.
Prior art systems for estimating voice quality do not properly support Service Level Agreements. Telephone service providers employing Voice over IP technology are desirous of offering Service Level Agreements in which they provide guarantees of voice quality, network availability and price. In order to properly implement such Service Level Agreements it is preferable to monitor every call and to record information on voice quality within call records.
Prior art systems do not support packet video systems, which also suffer from similar degradation due to the inability of the video decoder to filly reconstruct an image if the data is incomplete. Video compression systems typically employ motion coding in which the differences between an image and the previous image are transmitted. Errors can therefore be propagated through a series of subsequent images. The subjective effects of packet loss depend on the statistical distribution of lost packets and therefore it is desirable to consider the likely frequency of occurrence of multiple successive lost packets when estimating subjective video quality
Accordingly there is a need to provide a method of estimating subjective voice quality within a packet voice system that incorporates means of determining the loss in subjective quality due to a high rate of packet loss within a short time period.
Furthermore there is a need to provide a method of estimating subjective image quality within a packet video system that incorporates means of determining the loss in subjective quality due to a high rate of packet loss within a short time period.
Furthermore there is a need to provide a means of estimating subjective quality within a packet multimedia communications system that can determine said estimated subjective quality for every multimedia call in progress.
Furthermore there is a need to provide a means of estimating subjective quality within a packet multimedia communications system that can determine said estimated subjective quality for every multimedia call in progress and record said subjective quality within a call record database.
Furthermore there is a need to provide a means of estimating subjective quality within a packet multimedia communications system that is of low implementation complexity and can be installed in the form of a software addition to existing Voice over IP end systems.
The present invention provides an improved means of estimating subjective quality in packet multimedia communications systems wherein said communications system is presumed to have a low packet loss state and one or more high packet loss states and the statistical distribution of time spent in each state is determined thereby to predict the degradation in subjective quality caused by said packet loss and this information is combined with estimated degradation in subjective quality due to other communications system impairments in order to provide an estimated subjective quality measure for said multimedia communications system.
Accordingly, besides the objects and advantages of the present invention described above, several objects and advantages of the present invention are:
(a) to provide a method for estimating the subjective or perceptual quality of a multimedia communications system which considers the effects of xe2x80x9cburstyxe2x80x9d packet loss or short periods of high packet loss interspersed with periods of low or zero packet loss;
(b) to provide a method for estimating the subjective or perceptual quality of a multimedia communications system which considers the relative time at which different measured network impairments occur;
(c) to provide a distributed system for estimating the subjective or perceptual quality of a multimedia communications system at the conversion points of the multimedia communications system;
(d) to provide a distributed system for estimating the subjective or perceptual quality of a multimedia communications system which does not increase network traffic by requiring test messages to be sent in order to perform said estimation process;
(e) to provide a distributed system for estimating the subjective or perceptual quality of a multimedia communications system in which the subjective or perceptual quality is estimated on a per call basis and incorporated into a call record;
(f) to provide a method for estimating the subjective or perceptual quality of a packet voice connection;
(g) to provide a method for estimating the subjective or perceptual quality of a packet video connection;